Visualization of image transformation

ABSTRACT

A method is provided comprising: obtaining first and second datasets representative of first and second images of an object at different times, respectively; obtaining a deformation field, representative of changes between the first and second data sets, by performing a rigid or non-rigid registration; generating one or more masks and/or segmentations for selecting elements of the first image; selecting elements of the first image; transforming the first dataset using the deformation field to project the selected elements onto the second image; visualizing the deformation field or previously specified portions thereof; processing the deformation field or previously specified portions thereof to obtain data representative of different predetermined types of deformation; and visualizing the deformation field or one or more selected portions thereof, thereby to visualize the predetermined types of deformations separately and to enable a differentiation between changes in the patient and changes, in particular errors in the patient&#39;s position.

BACKGROUND OF THE INVENTION

Methods are known, for example irradiation therapy methods, which exposeareas of a patient with doses of irradiation. Such therapy is usuallypreceded by an irradiation planning session in which the region anddosage of irradiation is determined. This may involve obtaining CT imagedata of the patient. These steps may be repeated to examine the progressof the therapy. To do this successive CT images of the patient aregenerated, compared and analysed. Problems that may arise in thisprocess include movements of patient, patient respiration, and sizechanges of relevant portions of the patient, in particular the tissue tobe treated or tissue that is not to be exposed.

WO 2009/042952 A1 describes an irradiation method using deformable imageregistration. The method includes obtaining a first image, obtaining asecond image, determining a deformation field using the first and secondimages, and determining a transformation matrix from the deformationfield. The transformation matrix may be used to position the patientrelative to the radiation source to compensate for rotation andtranslation movements of the patient between the first and secondimages.

SUMMARY OF THE INVENTION

The present invention is defined in claim 1. Features of preferredembodiments are recited by the dependent claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a CT data set from the planning state, (a) and (c)with 3 superimposed structures (salivary glands and irradiation ortumour region) visualized as coloured contours and surfaces. (b) and (d)illustrate the superimposed dosage resulting from the planning shown ascolour encoded image (b) and with colour encoded isocontours (d).

FIG. 2 illustrates different visualisation possibilities of adeformation field, (a) with a colour encoded image representing theabsolute value (magnitude or amplitude) of the 3D vectors on current CTplane overlaid with global transparency on the original CT data set. (b)is the colour encoded isocontour representation of (a) withouttransparency. (c) and (d) are 2D projections of some 3D deformationvectors of current CT plane onto its plane wherein resulting 2D vectorsare displayed as (c) lines and (d) as amplitude dependent magnifiedarrows. Also the colour in (c) and (d) refers to the local 3Ddeformation magnitude. Subfigure (e) is like (c) but shows only localtransparency and (f) displays (e) as colour encoded image.

FIG. 3 illustrates a sagittal visualization of a deformation fieldoverlaid on the CT data set (a) with global opacity and (b) withtransparency of small magnitudes. (c) and (d) represents a filtereddeformation field wherein (c) only show deformations of structuressmaller scale without “distracting” coarse ones. (d) displays thiscoarse filtered deformation caused by errors in patient-positioning(neck curvature change). Globally these visualizations can beinterpreted as a change of the oesophagus and a patient positioningerror. (e) and (f) illustrates a 3D visualization of the deformation onthe planes whereby (f) is zoomed.

FIG. 4 illustrates a statistical evaluation of the deformation fieldwith common statistical parameters like mean, standard deviation, rootmean square, etc. Additionally the colour coded histogram of thedeformation magnitude is painted wherein (a) refers to the entirepatient and (b) to the irradiation (tumour) region. Globally the numberof small deformations is larger than the number of large deformations;in the tumour region the changes have a mean value of 5.69 mm. (c) showsthe volume change of different regions.

FIG. 5 illustrates a workflow in accordance with an embodiment of theinvention.

DETAILED DESCRIPTION OF THE DRAWINGS Introduction

Morphologic changes during radio oncological therapy cause changes inthe dosage distribution as compared to the dosage distribution at thetime of the initial radiotherapy planning. Such changes are quantifiedby applying the original irradiation plan onto data sets (e.g. CT or MRTimages) that are obtained at a later stage. For this purpose thecontours are applied by means of non-rigid image registration.

The morphology of the irradiated volume is exposed to various influencesduring the course of a series of treatments: shrinking or swelling oftissue, curvatures and torsions due to incorrect positioning or supportof the patient. However, the choice of adequate corrective measures inadaptive radiotherapy (ART) requires knowledge of the type and magnitudeof these sources of error.

Accordingly, in an embodiment the invention takes into account changesin the dosage distribution in the target volume and organs at risk, andstatic parameters in transformation matrixes in order to better judgethe type of transformation and to make better predictions of dosimetricchanges.

In particular, in an embodiment of the invention there is provided asoftware-implemented method for non-rigid image registration of CT datain selected regions of a patient, e.g. the ENT region. The methodcomprises calculation of transformation vectors representing translationand deformation of tissue between different data sets (CT images).

The transformation matrixes obtained by non-rigid registration ofCT-based data sets are used to adapt the planning contours,parameterized and statistically evaluated. After applying the originalirradiation plan onto the subsequent CT, the correlation of morphologicchanges and dosage changes can be analyzed. Besides diverse statisticalparameters the basis for an appropriate assessment of the data by anexpert is the visualization of the deformation itself. Severalrepresentations of the deformation field and the transformed datasupport the interpretation of the modifications of the body.

In order to evaluate morphologic changes, voxel data of thetransformation vector field preferably undergoes filtering (e.g. bandpass, smoothing, etc.) and statistical processing, e.g. on the basis ofmean values, standard deviations and vector lengths. In particular,morphologic changes, represented by mean vector lengths in filteredtransformation vector fields, may be correlated with dosimetric changes.The transformation matrixes enable a visualisation of deformations ofthe entire scan volume (position inaccuracies) by using global filtersor a visualisation of local size reduction and deformation processes byusing local filters.

Thus, it is possible to analyze morphologic changes during a series ofirradiation treatments and to non-rigidly register contours onto acurrent CT dataset. Thereby ART can be assisted, i.e. the adaptation ofthe irradiation and optimisation of the dosage plan in response tochanges in the shape and position of the irradiated volume. Thus, theanalysis of the deformation as result of the non-rigid registrationenables a specific correction depending on the nature of the morphologicchanges.

DESCRIPTION OF A PREFERRED EMBODIMENT

In an embodiment of the present invention there is provided asoftware-implementable method for improving the workflow and therapy inthe field of irradiation-based tumour treatment. As in conventionalsystems, the method comprises generating CT images of patients to betreated, and planning the treatment on the basis of such images. Theplanning maybe manual, semi-automatic or automatic and involves themarking of regions of interest in the CT images (e.g. the tumour region,salivary gland etc. See FIG. 1). Thereby the irradiation system can bealigned in order to directly irradiate the tumour without undue exposureof important organs such as the salivary gland. The irradiation is thenperformed over a period of weeks or even months in successive sessions,wherein the anatomy of the patient may change (patients may lose weight;organs change their size and position; patients have differentpositions; etc.). Since a new planning (drawing of contours) can bedifficult and time consuming, this is to be avoided unless there aresignificant changes in the patient's anatomy. This may result ininaccurate irradiation.

The present invention aims to provide a tool to distinguish between achange in position and an actual deformation. Thereby, unnecessaryre-planning steps can be avoided, because change in a patient'spositioning can be addressed by re-positioning the patient, whereas onlyactual deformations require a re-planning.

According to an embodiment of the present invention, a new dataset(usually CT image) is generated at regular intervals. By comparing achronological staggered CT image(s) with the original CT image on whichthe planning was based, the planning contours can be automaticallyadapted to the current data set. By detecting the necessity of are-planning, the accuracy of the irradiation can be optimised, resultingin higher chances of successfully treating the patient.

In addition, other data such as dosage distribution (e.g. to what dosagehas the salivary gland been exposed, see also FIG. 1b ) can betransformed onto more recent CT data. Thereby, the sum of dosage data ofeach irradiation session becomes more accurate.

When comparing the planning CT with the current CT, a so-callednon-rigid registration is performed. Thereby, a deformation (vector)field is generated that deforms the planning CT so as to substantiallyobtain the current CT. Using the deformation field, the contours and/ordosage is transformed and applied to the new (current) data set.

Subsequently, instead of discarding the deformation field as usuallydone, it is visualised in order to display the position, magnitude andquality of changes. For this purpose the rigid portion of thedeformation field (i.e. the portion associated with rotations andtranslations) is not displayed, as this does not contain any informationthat is sought at this stage. (Such information may reflect aninaccurate positioning of the patient, which may be useful at some otherstage. However, in terms of evaluating the therapy, only the non-rigidportion is relevant).

Different methods may be used to visualize the vectors of thedeformation field. In the present case, the deformation fields represent3D vectors. As it is preferred to display 2D sections of CT datasets,the 3D vector datasets are intersected with a 2D plane (axial, coronal,saggital, oblique), thereby generating a 2D matrix/image containing 3Dvectors (see FIGS. 3e and f ). However, it is preferred not to displaythese 3D vectors. Instead, in embodiments of the invention threealternative visualisation methods are implemented:

-   -   1. Absolute value (magnitude) of the 3D deformation vectors (see        FIG. 2a )    -   2. Colour-coded isocontours of 1.    -   3. Projections of the 3D vectors onto the 2D intersection place        (resulting in 2D vectors) and simultaneous colour-coding by        means of the new 2D magnitude or original 3D vector magnitude.        Also the thickness of the arrowhead of the drawn vectors (not        only the length) can be changed due to their magnitude.        Preferably, only a fraction of the vectors is displayed (e.g.        every x^(th) vector), see FIG. 2b ).

For physicians it is important to be able to distinguish betweendifferent kinds of deformations, e.g. has a patient been badlypositioned by the physician (e.g. different neck curvature in differentCT images), has an organ moved or has it changed its size. This ishardly visible from raw deformation fields. Accordingly, in anembodiment of the present invention, filtering in accordance with thestrength (deformation magnitude, see FIG. 3b ) and/or the scale(object/structure deformation's spatial size, see FIGS. 3c and d ) ofthe deformations is performed. Thereby, it is possible to selectivelydisplay different types of changes e.g. “strong” movements (e.g.shoulder movements) or minor movements (e.g. organ changes).

The deformation strength (small and large deformations) and scale(coarse and fine structures) can be adjusted separately e.g. acrosscontinuous intervals (minimum and maximum strength and scale), i.e. todisplay only deformations of objects at the size of a specific organ. Inthis case it is possible to filter in accordance with the interestingorgan and with interesting amplitudes (e.g. position errors and smallvector lengths <1 mm are currently uninteresting and undesired invisualization).

According to an embodiment of the invention, the planning CT istransformed by means of the filtered deformation field. This is doneiteratively on the basis of different deformation scales. Thereby a “3Dvideo” may be generated in which initially coarse spatial structurechanges are displayed (neck movements, see FIG. 3b ), followed by finespatial structure movements (organ movements, see FIG. 3a ). Thus, thevideo successively displays different kinds of changes wherein the timeaxis represents the scale or size of moving/deforming organs

Also, in an embodiment of the invention, a histogram of the deformationvectors is displayed (see FIG. 4a ), and to restrict this to a selectedregion (e.g. a tumour region, see FIG. 4b ).

According to an embodiment of the invention, inverse mapping is employedin order to transform volume data (such as contours). That is, vectorsare calculated that point from the current data set to the original dataset. This may be done through image registration. However, for thepurposes of visualisation it may be more intuitive to display vectorsthat point from the original dataset to the target (current) dataset.This may be done over a second registration, per vector field inversionor by registration algorithms which generate the forward and the inversedeformation field simultaneously. This forward deformation field canalso be used to transform continuous point data instead of volume data.To eliminate rigid content from the non-rigid transformation field (forvisualization) it is also possible to make a rigid registration stepbefore the non-rigid registration itself. This way, point data can betransformed while both deformation fields can be visualized.

It may be required that the volume data represent arrays of 2D polygonsthat reside in the same spatial plane as the CT data. For this reason asuitable transformation may be undesirably complicated (e.g. transformedpolygons are no longer in the same spatial plane as the z-axis, i.e. newpolygons must be generated). To address this, in an embodiment of theinvention, a new volume is generated from the 2D curves, and the newvolume is transformed and intersected with the resulting new planes. Inaddition, image processing steps to smoothen frayed regions may beapplied.

In accordance with an embodiment of the invention, the method comprisesdetermining a forward deformation field, a transformation matrix, and aninverse deformation field. The forward and inverse deformation fieldsare processed to eliminate a rigid portion thereof attributable torotation and/or translation of the object (patient).

In particular, first a segmentation of 1^(st) and 2^(nd) CT images ismade to obtain two input images and two binary segmentation images. Fromthese four images the transformation matrix (4×4 matrix containingtranslation and rotation) is determined. With the transformation matrix,the two input images and the two segmentation images, the forwarddeformation field and the inverse deformation field are determined.

More particularly, first the transformation matrix is determined, andsubsequently the deformation field and the inverse deformation field aredetermined. The transformation matrix is also used for imagetransformation (together with the inverse deformation field), as onlythe transformation matrix contains information representative ofmovements caused by translation or rotation of the patient. Furthermorethe transformation matrix, the forward (for continuous point data) andthe inverse deformation fields (for volume data) are used for RTstructure transformation.

In this embodiment, the deformation field or the inverse deformationfield on its own is insufficient for the purpose of objecttransformation. In other words, at least the transformation matrix andone of the deformation fields is required.

FIG. 5 illustrates a workflow in accordance with an embodiment of theinvention. In a first step, a 1^(st) CT image is generated. In a secondstep, an irradiation planning is performed on the basis of the 1^(st) CTimage. In a third step, the irradiation is performed. In a fourth step,a second CT image is generated. In a fifth step, the 1^(st) and 2^(nd)images are processed, as described above.

FURTHER EMBODIMENTS

According to an embodiment there is provided a method comprising:obtaining at least first and second CT images of an object, inparticular a patient or a portion thereof; performing a non-rigidregistration of the first and second images to determine a deformationfield describing a deformation of the object from the first CT image tothe second CT image; using the deformation field to determine atransformation of first image portions associated with the first CTimage into second image portions associated with the second CT image,wherein the first and second image portions may represent selectedcontours and/or dosages, in particular contours of organs or irradiationregions and irradiation dosages, respectively; and generating one ormore images to visualise the deformation field, wherein the one or moreimages may be superimposed on the first and/or second CT images.

The method may further comprise processing the deformation field toeliminate a rigid portion thereof attributable to rotation and/ortranslation of the object.

The method may further comprise generating a transformation matrixdescribing a transformation of the first CT image into the second CTimage; determining said deformation field from the transformationmatrix; and additionally using the transformation matrix to determinethe transformation of the first image portions associated with the firstCT image into the second image portions associated with the second CTimage.

The method may further comprise determining an inverse deformation fielddescribing a deformation of the object from the second CT image to thefirst CT image; and additionally using the inverse deformation field todetermine the transformation of the first image portions associated withthe first CT image into the second image portions associated with thesecond CT image.

The method may further comprise making a segmentation of the first andsecond CT images, thereby generating first and second segmentationimages; determining said transformation matrix from the first and secondCT images and the first and second segmentation images; and determiningsaid deformation field and said inverse deformation field from the firstand second CT images, the first and second segmentation image, and thetransformation matrix.

Preferably, the visualisation step comprises generating a colour-codedrepresentation of the absolute value of each of the entries of thedeformation field.

Preferably, the deformation field contains 3D vectors, and wherein thestep to visualise the deformation field comprises: projecting each ofthe 3D vectors onto a 2D sectional plane; and generating a colour-codedrepresentation of the projected 2D vectors based on the absolute valueof the corresponding 3D vectors.

Preferably, the step to visualise the deformation field comprisesdisplaying a fraction of the 2D vectors selected in accordance with apredetermined selection criterion.

The method may further comprise filtering entries of the deformationfield depending on the degree of deformation, in particular entriesrepresentative of selected types of deformation such as movements orchanges of the object's anatomy.

The method may further comprise filtering entries of the deformationfield depending on the size of the region affected by the deformation,in particular entries representative of a deformation that affects aregion whose size is not within a predetermined size range.

The method may further comprise iteratively transforming the first imageon the basis of filtered entries of the deformation field to obtain asuccession of images representing different magnitudes of deformation.

The method may further comprise generating and displaying a histogram ofthe entries of the deformation field, in particular in respect of aselected region of the object or first or second images.

Preferably, the first image represents an array of 2D polygons, whereinthe method comprises: generating a volume from the 2D polygons;transforming the volume; and intersecting the volume with one or morepredetermined planes.

It will be appreciated that the above described embodiments aredescribed as examples only, and that modifications to these embodimentsare included within the scope of the appended claims.

What is claimed is:
 1. A method of evaluating irradiation therapy of apatient, the method comprising: obtaining first and second datasetsrepresentative of first and second CT images of the patient at differenttimes, respectively; obtaining a deformation field, which includes threedimensional vector field entries, representative of changes in one ormore regions of the patient between the first and second datasets,wherein the deformation field is generated by performing a rigid ornon-rigid registration; subsequent to completion of the rigid ornon-rigid registration, selectively filtering the entries of thedeformation field depending on size of the one or more regions, wherein,for each of the one or more regions whose size is not within a selectedsize range, the filtering comprises band pass filtering to filter theentries representative of the changes that affect the respective regionwhose size is not within the selected size range; and generating asuccession of images representing different magnitudes of deformationbased on the first CT image and the filtered entries of the deformationfield, thereby to visualize deformations in enabling differentiationbetween changes in the patient's anatomy and changes, in particularerrors, in the patient's position.
 2. The method of claim 1, furthercomprising: generating one or more masks and/or segmentations forselecting elements of the first CT image, wherein the selected elementsmay represent contours, points, and/or dosimetric image portions.
 3. Themethod of claim 1, further comprising visualizing said deformationsseparately.
 4. The method of claim 1, further comprising: usingalgorithms to simultaneously generate forward and backward deformationfields; obtaining a rigid transformation attributable to rotation and/ortranslation of the patient and a non-rigid transformation, attributableto actual deformations inside the patient; providing the deformationfields separately and as a combined deformation field; and transformingthe first dataset using the rigid transformation field and the non-rigidtransformation field, or the combined deformation field to project theone or more regions whose size is within the selected size range ontothe second CT image.
 5. The method of claim 1, further comprising:transforming segmentation images with a rigid, non-rigid or combinedtransformation, for example a deformation field, obtained by aregistration from a first image to a second image, wherein thesegmentation image is represented by a first set of planar 2D polygonsin 3D space; generating a 3D volume (3D image matrix) from the set of 2Dpolygons; transforming the volume using predetermined matrixtransformation algorithms; inserting the volume with planar planes; andextracting the contours from the intersected planes to obtain a secondset of planar 2D polygons.
 6. The method of claim 1, further comprising:generating one or more statistical evaluations, in particularhistograms, boxplot, mean, variance, of the deformation field or partsof it.
 7. The method of claim 1, further comprising: visualizing theimages as single images or as an image sequence; and superimposingresulting images, contours or points on the datasets.
 8. The method ofclaim 1, further comprising: generating a forward and a backward/inversetransformation matrix describing a transformation of the first CT imageinto the second CT image or vice versa; determining said deformationfield from the transformation matrices; and additionally using thetransformation matrices to determine the transformation of first imageportions associated with the first CT image into second CT imageportions associated with the second CT image or vice versa.
 9. Themethod of claim 1, wherein the generating the succession of imagescomprises generating a colour-coded representation of the absolute valueof each of the entries of the deformation field.
 10. The method of claim1, wherein the generating the succession of images comprises: projectingeach of the entries onto a 2D sectional plane to generate a projected 2Dvectors; generating a colour-coded representation of the projected 2Dvectors based on the absolute value of the corresponding entries;generating isocontours based on the magnitude of the projected 2Dvectors; and displaying the projected 2D vectors with differentglyph-types.
 11. The method of claim 10, wherein the generating thesuccession of images comprises displaying a fraction of the projected 2Dvectors selected in accordance with a predetermined selection criterion.12. The method of claim 1, further comprising: generating and displayinga histogram of the entries of the deformation field, in particular inrespect of the one or more regions of the patient or the first or secondCT images.
 13. The method of claim 1, wherein the band pass filteringincludes filtering to filter the entries representative of the changesthat affect the respective region whose size is larger than the selectedsize range while retaining entries representative of the changes thataffect the respective region whose size is within the selected sizerange.
 14. A non-transitory storage medium comprisingcomputer-executable code that, when executed by a computer system,causes the computer system to perform the method of claim 1.